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Introduction.  Because  of  the  high  cost  of  testing,  many  large 
weapon  systems  cannot  be  tested  over  the  full  range  of  possible  battle¬ 
field  parameters.  As  a  result,  the  developer  and  the  reviewing  authori¬ 
ties  have  come  to  rely  on  system  simulation  to  demonstrate  the  system 
capability  over  the  range  of  untested  parameters.  These  simulations 
also  are  useful  to  investigate  the  change  in  performance  resulting  from 
possible  subsystem  modifications.  In  some  important  programs,  the 
lovernment  relies  on  simulations  of  competing  systems  to  indicate  the 
relative  performance  of  these  systems  in  situations  for  which  no  tests 
have  been  made,  although  of  course,  simulations  such  as  these  have  been 
validated  as  much  as  possible  by  system  tests.  In  these  instances  the 
procurement  decision  rests  heavily  on  the  validity  of  the  system  simula¬ 
tions.  Consequently  the  need  arises  for  a  generally  accepted  procedure 
which  is  undoubtedly  fair  to  each  contractor  and  provides  the  maximum 
amount  of  objective  judgment  about  the  validity  of  the  simulation.  In 
any  such  procedure  the  Government  must  be  able  to  rapidly  evaluate 
simulations  furnished  from  a  variety  of  sources. 

The  procedure  must  be  workable  and  economical  --  that  is  it  must 
apply  a  lot  of  leverage  to  the  problem  with  regards  to  manpower,  — 
computer  programmers  and  engineers  --  the  cost,  --  computer  running 
time  and  validation  experiments  --  and  elapsed  time.  Implicit  in  this 
discussion  is  that  planning  for  system  simulation  validation  must  be 
completed  before  the  first  system  RFQ  is  issued. 

Many  methods  are  used  for  system  simulations:  Monte  Carlo,  analog, 
hybrid,  and  digital  simulation  of  differential  equations.  A  variety  of 
special  and  general  purpose  programs  are  available  for  the  simulator's 
use.  Among  then  are  "SPERT",  "ACSL,"  and  HIT  PRO."  The  problem  for 
the  user  of  these  simulations  comes  when  he  needs  to  compare  theory 
with  experiment  and  asks  the  questions:  How  good  is  the  theory?  Is 
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the  agreement  between  theory  and  experiment  good  enough  to  validate  the 
simulation?  (As  an  example  of  these  questions  see  Pastrick  (1,  2).) 

Another  question  to  be  considered  is:  Could  it  be  that  the  experiment 
was  defective  in  any  way? 

Many  simulations  have  not  been  prepared  in  such  a  way  so  that  they 
can  be  used  to  answer  these  questions.  In  the  first  place,  the  simula¬ 
tions  are  not  designed  to  adjust  parameters  to  fit  data.  In  the 
second  place,  the  system  itself  may  be  so  complex  that  the  computing 
time  for  complete  system  simulation  is  so  long  that  adjusting  the 
parameters  to  achieve  a  better  fit  between  simulation  and  experiment  is 
not  feasible.  Thus  a  new  procedure  is  needed  to  combine  theory  and 
experiment . 

The  procedure  suggested  by  this  paper  is  the  use  of  the  least  chi 
square  computer  program  to  simulate  the  major  subsystems  of  a  system 
simulation  and  validate  it  against  test  data. 

Criteria  for  comparison,  and  iteration  procedure.  In  fitting  data 
to  non-linear  models  of  system  performance  such  as  systems  of  differen¬ 
tial  equations,  the  usual  criteria  is  the  iterative  minimization  of  the 
sum  of  the  squares  of  the  residuals.  Other  criteria,  such  as  generalized 
least  squares  have  also  been  considered  and  demonstrated  (3).  As 
Aitken  (4)  noted  with  respect  to  generalized  least  squares,  the  criteria 
to  be  used  are  a  matter  of  choice.  In  other  words,  we  are  free  to 
decide  whether  least  squares  is  the  best  criteria  for  our  purpose.  A 
particular  concern  with  the  ordinary  least  squares  procedure  is  whether 
the  residuals  are  consistent  with  being  drawn  from  a  random  sequence. 

Many  tests  have  been  devised  (5)  for  this  purpose.  One  test  of 
special  interest  in  this  paper  is  the  Box-Pierce  (6)  test  which  is  the 
sum  of  the  squares  of  the  autocorrelation  coefficients  divided  by 
their  variances.  A  typical  term  is  tr(>*  v'j-1 

Given  all  these  tests,  no  way  had  been  devised  to  adjust  the 
parameters  to  better  satisfy  the  data  until  it  was  proposed  that  this 
criteria  be  combined  with  least  squares  to  obtain  a  new  criteria: 
least  chi-square  (Moore,  7,  8,  9).  By  finding  the  parameters  which 
minimize  chi-square,  the  probability  is  maximized  that  the  residuals 
come  from  a  population  with  a  given  variance  ,  and  from  a  random 

sequence.  The  variance  must  be  independently  determined  from  theory  or 
measurement  as  the  measurement  error. 

Thus,  a  probability  can  be  generated  from  the  computed  chi-square 
which  permits  the  statistician  and  decision-maker  to  compare  the  "good¬ 
ness  of  fit"  of  the  simulation  of  several  quite  different  systems.  In 
this  way  a  direct  comparison  of  the  validity  of  the  simulations  can  be  made. 
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The 


figure  of  merit,  (chi-square  total)  is  the  sum  of 

th*»  Rnv-Poi  rrp  niimh«>r  /  V*  i  V  /m  .11 


and  5^  ,  the  Box-Peirce  number 

Derivation:  We  will  follow  the  procedure  and  most  of  the  notation 
of  Aitken  (4)  for  generalized  least  squares: 


Let  the  representation  of  the  vector  of  data: 

U«  (u(X,J.  u  (X| ) .  ...  u(x  n 
n 

by  the  theoretical  vector,  be: 

v*{y<*,).  y(*«).  ...  y(*  )) 

n 


V*w- 


Let  0*  dejrote  a  colum) vector  of  k  +  1  coefficients  independent 
of/such  that: 

«,♦.  e,*.  ...  ejt|} 

Define  the  matrix  P*  as  the  matrix  whose  ith  row  is 


ay,*  »V  ...  fV 
aiT"  ae,  aer 


(The  asterisk  symbol  *  will  be  used  to  indicate  an  estimate  of  the 
indicated  sysbol  where  convenient.  However,  it  will  not  be  used  on 
complex  expressions  involving^*-  w* ,  and  r.  because  of  typographical 
difficulties) .  r  * 


In  this  expression  is  defined  as  follows: 
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In  these,  the  subscript  "j"  indicates  a  unit  value  in  each  of  the  ith 
rows  and  (i  +  j)th  column. 


If  is  the  variance  of  ,  then 

V*V,d’d*f.i  ri,/vi 

and:  /W) 

(Note  the  difference  between 
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On  differentiating  (  )  with  respect  to  (  ©* )  and  substituting 

W)  *  p*  (#«*)  -  u*. 

as  an  estimate  of  the  increment  of  the  resic’itals  needed  to  minimize^.*  , 
the  algorithm  for  C/®!Jbecomes : 

|oe*l  •  [p*'  r  p*i-‘  p*‘  r  u*. 

WM#«4T; 

2r  V-1 

«,*  - -i  i 

Cd)'(d)/o  *  -  j£  (r,)*  v.~' 
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If  P  equals  I,  the  expression  for  OtTl  reduces  to  c 
which  is  the  same  as  the  algorithm  for  ordinary  non-linear  least  squares 
used  in  such  computer  programs  as  provided  by  both  IBM  and  CDC  libraries 
as  well  as  in  SAAM-27. 

By  inspection,  r  replaces  P*1 in  the  ordinary  expression.  To 
modify  the  ordinary  expression, f*  is  computed.  P*’  is  postmultiplied 
by  P  ,  and  the  product  placed  in  the  computer  memory  where  P* ’  is 
normally  stored.  it*  is  substituted  for  dd *  whereever  it  occurs  and 
no  further  change  is  need  in  the  iteration  procedure. 

These  expressions  have  been  programmed  into  the  Simulation  And 
Analysis  Modeling  (SAAM-27)  program  of  Berman  et  al,  (10,  11)  as  indi¬ 
cated  above,  multipying  P*' ,  by  p  ,  and  letting  the  program  proceed  from 
that  point.  The  usual  iteration  continues.  The  computer  program 
resulting  from  this  change  has  been  designated  for  control  purposes  as 
SAACH,  and  has  been  tested  on  the  CDC  6600  at  ARRADCOM,  Dover,  to  deter¬ 
mine  the  following  questions: 

1.  How  much  change  is  there  in  the  final  parameter  estimates? 

2.  What  change,  if  any,  is  there  in  the  number  of  iterations? 

3.  What  change  is  there  in  the  time  per  iteration? 

Four  problems  of  different  origin  and  which  use  different  mathe¬ 
matical  models  have  been  run  on  the  SAACH  program  to  answer  the  above 
questions.  In  the  first  example:  Gun  Chamber  Pressure  Waves,  the 
mathematical  model  used  is  the  superposition  of  two  pressure  waves 
generated  by  analytic  models  in  the  program,  with  the  adjustment  of  up 
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to  eight  parameters  to  obtain  the  best  fit  to  observed  data.  In  the 
second  example,  an  aircraft  control  system  simulation,  the  mathematical 
model  is  a  set  of  four  linear  differential  equations,  simulating  the  Yaw 
Damper  system  on  an  aircraft.  These  equations  were  solved  by  a  special 
procedure  developed  for  SAAM-27  by  Berman  et  al.  (12),  with  up  to  four 
adjustable  parameters.  In  the  third  example,  a  biomedical  problem 
furnished  as  a  test  case  by  Miss  Rita  Straub  of  Brookhaven  National 
Laboratory,  the  mathematical  model  was  a  set  of  seven  coupled  linear 
differential  equations  with  five  adjustable  parameters;  this  was  solved 
by  the  same  method  as  used  in  the  second  case.  In  the  fourth  and  final 
example:  KEWB  Kinetics,  a  simulation  of  the  nuclear  reactor  transients 
of  the  Kinetic  Experiment  Water  Boiler,  the  mathematical  model  was  an 
extremely  non-linear  set  of  coupled  differential  equations  as  described 
by  Hetrick  and  Gamble  (13).  These  equations  were  integrated  by  the 
fourth  order  Runge-Kutte  integration  procedure  of  SAAM-27,  with  only 
one  adjustable  parameter.  The  results  of  these  analyses  were  discussed 
in  detail  at  the  1979  Design  of  Experiments  Conference  (9).  Abbreviated 
discussions  of  the  result  of  each  problem  follow. 

Gun  Chamber  Pressure  Waves .  Unusual  pressure  waves  suggestive  of 
an  acoustic  wave  superposed  on  the  normal  gun  chamber  pressure-time 
curve,  have  occurred  in  tests  of  the  XM211  propellant  charges  at  zone  3 
for  the  M101  projectile  in  the  155mm  gun,  (Knutelski,  (14)).  The 
mathematical  model  used  was: 


P  «  Pj  exp  {  -  it-ti)*/2 o12  } 

+P2  exp  f  -  (t-t2)V2ot*J  X  sin£2irf  (t-t3)  + 

Three  parallel  cases  were  computed  once  the  fit  was  good  enough  to 
permit  iteration  with  different  ranks  of  autocorrelation.  Because  of 
computing  difficulties  which  arose  when  trying  to  converge  on  six  or 
seven  parameters,  the  iteration  was  initially  restricted  to  four  para¬ 
meters:  Once  the  fit  was  good  and  had  converged  using  these  four  para¬ 
meters,  their  final  values  were  used  as  initial  values  for  a  six-parameter 
fit.  Finally,  all  eight  parameters  were  allowed  to  vary. 

Two  results  of  this  series  of  analysis  are  plotted  in  Figs  1,  and 
2.  The  case  numbers  are  BGK-3. 30356301-0,  and  3.30356511-5.  The  first 
has  no  autocorrelation  coefficients;  the  second,  5,  a  third,  (not 
shown),  10.  The  parameters  for  these  cases  are  given  in  Table  1,  (note 
that  the  last  three  digits  only  of  the  identifier  are  used  here).  Some 
parameters  are  quite  different  from  case  to  case. 
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The  apparent  fit  from  the  figures  is  best  for  the  case  of  five 
autocorrelations  given  in  Fig.  2.  The  fit  of  this  case  was  also 
better  than  that  for  10  autocorrelations  which  is  not  illustrated. 

The  value  of  the  sum  of  the  squares  of  the  residuals  as  shown  in 
Table  1  was  actually  much  smaller  for  cases  511-10,  and  301  than  for 
511-5  yet  the  fit  as  noted  above  was  not  as  good. 

The  last  row  of  Table  1,  gives  the  values  of  ^  the  experimental 
variances  assumed  for  these  cases.  These  were  arbitrary  numbers  taken 
as  example  only,  because  the  precision  of  the  measurement  system  is 
probably  much  greater  than  the  value  given;  i.e.,  the  variances  should 
be  smaller.  However,  if  smaller  values  were  used,  such  as  when  case 
511-10  is  compared  to  511-5,  the  weight  on  the  sum  of  the  squares  is 
greater  but  the  goodness  of  fit  appears  to  decrease,  thus  illustrating 
the  need  for  least  chi-square  iteration. 

Table  1  also  shows  the  effect  of  least  chi-square  in  terms  of 
number  of  iterations,  and  computing  time.  When  five  autocorrelations 
were  used,  as  in  case  511-5,  only  a  small  increase  in  number  of  itera¬ 
tions  is  found  and  a  moderate  increase  in  computing  time  as  compared  to 
301.  If  ten  autocorrelations  were  used,  as  in  511-10,  the  number  of 
iterations  increased,  and  the  time  increase  was  1.8  times  greater,  giving 
about  double  the  increase  in  time  for  double  the  number  of  autocorrela¬ 
tion  coefficients. 

Table  2  shows  the  autocorrelations  up  to  order  20  for  the  three 
cases.  The  values  of  Xj  ,  ,  and  X^  for  the  number  of  autocorrela¬ 

tions  used  (0,  5,  10)  is  shown  in  theiast  rows  of  this  table. 

Aircraft  Control  Systems.  A  typical  aircraft  yaw  damper  design  pro¬ 
blem  (15),  was  analyzed  to  illustrate  the  use  of  least  chi-square.  To 
optimize  the  design,  four  parameters  may  be  adjusted  to  give  the  best  fit 
to  a  desired  response  curve.  These  parameters  are  ^  ,  and  ^ 

These  correspond  to  the  parameters  L(0,4),  L(4,l),  L(4,l),  1.(4, 2)  and 
L(4.3).  A  vector  of  a  random  sequence  of^normally  distributed  errors 
from  a  population  with  variance  of  (.033)  was  added  to  the  data  vector 
to  simulate  the  effects  of  sampling  error;  this  may  be  considered  to 
represent  an  allowable  error  or  tolerance  in  fitting  the  curve. 

2 

The  value  of  was  set  at  (.033)  ,  six  autocorrelations  were  used 
for  the  problem  which  was  identified  as  C0NRLM  4.011-6.  Another  run 
was  used  on  the  same  data  with  the  standard  least  squares  algorithm. 

Fig.  3  shows  the  fit  obtained  for  the  data  and  is  typical  of  the  results. 
Table  3  showns  the  number  of  iterations  for  each  case.  It  took  4  itera¬ 
tions  for  the  ordinary  algoithm  to  converge,  and  only  two  for  the  least 
chi-sqare  algorithm  with  six  autocorrelation  coefficients  (CONRLM  4.011-6). 
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CASE 


UNITS 

SYMBOL 

301 

515-1 

511-10 

10  kpsi 

10  kpsi 
sec 

sec 

sec 

sec 

sec 

Hz 

;i 

2 

l3 

i1 

20.04 

4.146 

.057 

.056 

.0557 

.0027 

.00051 

327.6 

19.7 

33.3 

.0568 

.0564 

.0558 

.00279 

.000597 

344.4 

19.95 

3.998 

.0568 

.0565 

.0558 

.00270 

.000519 

361.08 

No  of 

Iterations 

18 

19 

21 

Computing 

Time  (sec) 

51.2 

58.0 

63.2 

Experimental 

Variance 

e 

NOT  APPLICABLE 

( -  45 ) 2 

(  -  3 1 7 ) 2 

TABLE  1. 

Parameters  Fitting  Pressure  Curve. 

CASE 

301 

511-5 

511-10 

ORDER 

I 

.111 

.6  80 

.696 

2 

.477 

.427 

.4*3 

3 

.286 

.247 

.248 

4 

.056 

.027 

•  013 

5 

-.113 

-.  136 

-.  160 

6 

-.246 

-.256 

-.291 

7 

-.303 

-.315 

- .  J46 

8 

-.322 

-.334 

-.361 

9 

-.307 

-.312 

-.342 

10. 

-.245 

-.240 

-.276 

11. 

-.130 

-.104 

-.153 

12. 

-.011 

.031 

-.035 

13. 

.069 

.130 

.046 

14. 

.137 

.217 

.123 

15. 

.  110 

.203 

.  106 

16. 

.075 

.166 

.085 

17. 

-,028 

.057 

.0004 

16. 

-.  14  J 

-.068 

-.090 

19. 

-.202 

-.149 

-  130 

20. 

-.235 

-  205 

-.145 

S<Jft 

»»  M 

S«e. 

57.9 

67.9 

S7.» 

62.96 

116.2 

'i 

- 

33.01 

62.4 

*T0T 

. 

95  97 

178.6 

fable  2  Autocorrr lotions  end  Chi-Square  for  final 
•o«lc  1  of  XH211  Pretturr  Oicillationt 
(X  2  bated  on  the  first  5  Auiocorrelat iont 
foi  Cate  511-5.  and  the  firtt  10,  for  cate 
511-10) 
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CASE 

4.012-0 

4.011-6 

HO.  of  ITER. 

4 

2 

1(0,4) 

17.70 

18.58 

1(4,1) 

53.11 

53.02 

1(6,2) 

1.094 

0.605 

1(6,3) 

6.204 

10.26 

*U)  l 

-.110 

2 

-.232 

3 

-.014 

4 

.068 

5 

-.124 

6 

.030 

Sum  of 

.03147 

.03138 

V 

28.90 

28.82 

X22 

2.77 

2.49 

«T* 

31.67 

31.31 

Table  3.  Results  of  Yaw  Damper  Calculations 

ITEM/ CASE 

KJE  1.0021-0 

KJE  1.0023-5 

1(1,1) 

.2155 

.2199 

1(2,1) 

.4527 

.4442 

U3.1) 

.0431 

.0431 

(.(4.1) 

.0252 

.0251 

US..) 

.0743 

.0824 

*(1) 

.122 

-.086 

R(2) 

.065 

-.052 

R(3) 

.009 

.034 

R(4) 

.227 

-.223 

R(S) 

.179 

-.168 

31.22 

31.24 

*2* 

3.99 

3.04 

*TOT 

35.21 

34.28 

Table  4. 

Results  of  Brookhaven  example  calculation. 

Autocorrelation,  X.,  and  XJ 

for  case 

KJE  1.0021-0 

computed  for  coa^arisoa. 

KWB  1.003-0  KVB  1.005-3  KWB  1.0007-6 

c  v i e  t  v 


5.310  X10 
-702 
.44 
.098 
-.204 
-.316 
-.235 


5.3183  X10 
.782 
.44 
.098 


5.262  X  10 
.786 
.453 
-.122 
-.170 
-.274 
.290 


Table  5.  Result*  of  Kinetic  Experiment  Water  Boiler  Calculations 
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The  times  for  one  iteration  were  8.2  and  8.5  sec  respectively.  (Part  of 
the  increase  in  time  for  the  least  chi-square  case  was  due  to  several 
attempts  in  both  iterations  to  improve  the  fit  by  reducing  the  step  size. 
As  shown  in  Table  3  the  parameters  L(0,  4),  L(4,2)  and  L(4,3)  appear  to 
be  different  by  significant  amounts.  (The  autocorrelations  for  case 
4.011-6  appear  well  within  the  random  range.)  The  normalized  sum  of 
squares  of  the  residuals  is  less,  as  expected,  for  the  ordinary  least 
squares,  Case  4.012. 

Brookhaven  Example.  A  sample  test  case  was  received  from 
Miss  Rita  Straub  of  Brookhaven  National  Laboratory.  The  exact  nature 
of  the  probem  was  unspecified  but  from  the  form  of  the  differential 
equations  it  appears  to  be  a  kinetic  problem  in  which  the  material  in 
component  one  decays  into  components  two  to  five,  and  component  two 
may  change  into  component  one.  Component  seven  is  composed  of  com¬ 
ponents  three,  four,  and  five.  Although  some  coupling  parameters  may 
actually  be  unknown,  they  were  assumed  known,  because  the  present 
version  of  the  program  will  not  iterate  either  type  of  linear  coupling 
parameter  with  the  least  chi-square  algorithm.  The  data  were  avail¬ 
able  for  the  amount  of  components  1,  3,  4,  and  6  as  a  function  of  time, 
(where  component  6  is  the  sum  of  components  1,  2,  and  5). 

Both  the  run  with  no  autocorrelations  and  the  run  with  5  corre¬ 
lations  (KJE  1.0023-5),  took  7  iterations  to  converge.  The  results  for 
the  two  cases  are  compared  in  Table  4.  Since  the  value  of  X^  ,  (31.24) 
is  large  compared  to  (3.04),  the  major  emphasis  in  this  case  was  on 

reducing  the  sum  of  squares,  and  thus  it  is  similar  to  the  case  run 
with  no  weight  on  the  autocorrelations.  As  would  be  expected,  there  is 
only  a  small  difference  between  the  final  values  of  the  parameters  of 
the  two  cases. 

Reactor  Kinetics  Example.  This  example  illustrates  two  things: 

First  the  use  of  the  least  chi-square  algorithm,  and  second  an  apparently 
good  fit  between  data  and  a  physically  incorrect  model.  Hetrick  and 
Gamble  (13)  proposed  a  non-linear  feed-back  term  proportional  to  the 
energy  in  the  reactivity  of  the  KEWB  reactor  to  describe  the  fit. 

Later  experiments  (16)  where  the  void  amount  was  inferred  from  measure¬ 
ments  and  where  the  thermal  effects  on  reactivity  were  also  carefully 
measured,  showed  that  shutdown  was  due  to  thermal,  not  void  effects. 

In  the  simulation,  the  effect  of  the  energy  on  void  formation  was  simu¬ 
lated  by  the  parameter  L ( 1 1 ,  1).  The  functions  correspond,  in  numerical 
order,  to  the  functions  used  in  the  simulation:  (1)  Nuclear  reactor 
power  level,  (2)  Mean  temperature,  (3)  Mean  void  volume,  ( 4 ) — ( 9 )  Delayed 
neutron  groups,  (10)  Not  used,  (11)  Energy  released  to  that  time.  The 
result  of  the  iterations  is  shown  in  Figure  4,  a  logarithm  plot  of 
theoretical  and  experimental  nuclear  power.  In  Table  5,  three  different 
cases  are  shown: 
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Case  1.003-0  was  ordinary  least-squares.  The  values  of  the  auto¬ 
correlations  and  chi-squares  are  shown  for  comparison  with  the  other 
two  cases.  Case  1.005-3  used  three  autocorrelations  with  a  small  ^alue 
of  the  experimental  variance  thus  resulting  in  a  large  value  of  X^  . 

Both  case  1.007-6  and  1.003-0  use  1  x  10  for  the  experimental  variance 
thereby  reducing  the  emphasis  on  the  sum  of  the  squares  of  the  errors. 

All  of  these  runs  took  four  iterations  to  converge. 

Cases  1.003-0  and  1.005-3  give  almost  exactly  the  same  results.  On 
comparing  1.003-0  with  1.007-6,  a  difference  is  found  in  the  value  of  the 
adjustable  parameter  L(ll,l).  The  value  of  chi-square  total  is  smaller  for 
1.007-6,  and  thus  this  result  would  be  chosen  over  that  of  the  other  case. 

The  value  of  the  chi-square  for  £he  Box-Pierce  number  is  much 
smaller  for  case  1.007-6,  although  X..  is  slightly  larger  for  the 

same  case--thus  illustrating  the  traae-off  between  getting  the  minimum 
as  in  ordinary  least  squares,  and  reducing  the  autocorrelations  as  in 
least  chi-squares.  The  data  for  Case  1.003  show  the  values  for  R(l)  to 
R(6)  for  comparison  purposes.  The  data  show  that  the  sum  squares 
does  not  increase  from  one  to  the  other  appreciably,  but  X^  ,  the 
Box-Pearce  statistic,  does  change  appreciably.  Each  of  the  calculations 
give  a  total  chi  square  which  is  too  large  to  be  consistent  with  the 
residuals  being  drawn  from  a  random  sample,  and  thus  would  have  given 
support  for  the  rejection  of  the  Hetrick-Gamble  model. 

Comparison  of  Computing  Time.  Table  6  summarizes  the  comparison 
of  the  number  of  iterations  to  converge,  and  the  computing  time.  The 
number  of  iterations  was  usually  about  the  same.  As  seen  in  the  last 
column  the  computing  time  is  comparable,  with  a  tendency  for  the 
computing  time  to  be  longer  for  least  chi-square  than  for  least  squares. 

The  relative  difference  is  greater  when  the  original  total  computing 
time  is  short.  This  just  means  that,  as  would  be  expected,  it  takes  a 
larger  fraction  of  the  computing  time  to  compute  the  matrix  F  and  post- 
multiply  into  P*'  for  cases  where  the  time  of  iteration  is  short. 

Conclusions .  Based  on  four  different  types  of  non-linear  theoretical 
models  for  data  analysis,  our  results  indicate  that: 

(1)  Least  chi-square  is  practicable  for  non-linear  analysis. 

(2)  Least  chi-square  gives  a  better  fit,  and  is  a  more  reliable 
iteration  procedure. 

(3)  The  computing  time  for  least  chi-square  is  longer  for  the 
models  which  use  less  computing  time,  but  because  the  convergence  of 
this  iterative  procedure  is  somewhat  better,  the  number  of  iterations 
(and  particularly  the  number  of  "tries"  per  iteration)  is  reduced,  thus 
keeping  the  total  computing  time  about  the  same.  Those  models  with  longer 
integrating  time  would  be  expected  to  benefit  more  from  least  chi-square. 
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(4)  In  validation  of  simulations  of  future  Army  systems,  the 
SAAM-27  computer  program  modified  for  least  chi-square  can  be  used  at 
various  stages  in  the  system  development.  First,  as  a  tool  to  simulate 
subsystems  and  compare  the  projected  performance  with  the  designer's 
simulation.  Second,  as  subsystems  are  built  and  tested,  they  can  be 
run  as  "hardware  in  the  loop"  and  the  test  data  used  in  the  least  chi- 
square  program  to  validate  the  computer  simulation  and  provide  system 
parameter  identification.  Because  no  programming  is  needed  to  run 
SAAM-27  on  a  variety  of  problems,  both  the  programming  time  and  the 
elapsed  time  is  greatly  reduced. 

By  planning  ahead  to  use  SAAM  in  the  validation  of  the  subsystem 
modeling  and  providing  the  needed  subsystem  tests,  a  Program  Manager 
can  reduce  the  time  and  effort  needed  to  validate  the  contractor's 
system  simulation  and  will  be  able  to  give  an  impartial,  knowledgeable, 
and  timely  evaluation  of  each  system. 
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Ho.  of  Adjustable  Rank  of  Audo-  No  of  Iters-  Time 
_ Parameters _ correlation _ tions _ (sec) 


Gun  Chamber  Pressure 

Curve.  8 

8 

8 

Yaw  Damper .  4 

4 


0 

18 

51.2 

5 

19 

S8.0 

10 

21 

63.7 

0 

4 

8.2 

6 

2 

8.5 

Biomedical  Test 
Case. 


5 

5 
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0  5  14.9 

*  5  15.2 

5  7  21.5 


Reactor  Kinet 
Experiment . 


1 

1 
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0 

3 
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4 

4 
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Table  6.  Comparison  of  computing  time  and  number  of  iterations. 
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